An intuitive user interface for visual sports coaching

This paper describes a dynamic multi-video user interface for sports coaching. It is intended that sports coaches could use this split screen to minimise and maximise multiple video streams of an athlete on one side of the split screen, while playing an additional video source on the other side of the split screen, such as a clip from a professional athlete. This split screen approach allows users to contrast movements in the athletes videos, with that of a professional. Users can also avail of the ability to use video overlays, text input and can also use screen capture technology to record the application display, so that an athlete can review a coaching session at later date

Multi-sensor human action recognition with particular application to tennis event-based indexing

The ability to automatically classify human actions and activities using vi- sual sensors or by analysing body worn sensor data has been an active re- search area for many years. Only recently with advancements in both fields and the ubiquitous nature of low cost sensors in our everyday lives has auto- matic human action recognition become a reality. While traditional sports coaching systems rely on manual indexing of events from a single modality, such as visual or inertial sensors, this thesis investigates the possibility of cap- turing and automatically indexing events from multimodal sensor streams. In this work, we detail a novel approach to infer human actions by fusing multimodal sensors to improve recognition accuracy. State of the art visual action recognition approaches are also investigated. Firstly we apply these action recognition detectors to basic human actions in a non-sporting con- text. We then perform action recognition to infer tennis events in a tennis court instrumented with cameras and inertial sensing infrastructure. The system proposed in this thesis can use either visual or inertial sensors to au- tomatically recognise the main tennis events during play. A complete event retrieval system is also presented to allow coaches to build advanced queries, which existing sports coaching solutions cannot facilitate, without an inordi- nate amount of manual indexing. The event retrieval interface is evaluated against a leading commercial sports coaching tool in terms of both usability and efficiency

Toward next generation coaching tools for court based racquet sports

Even with today’s advances in automatic indexing of multimedia content, existing coaching tools for court sports lack the ability to automatically index a competitive match into key events. This paper proposes an automatic event indexing and event retrieval system for tennis, which can be used to coach from beginners upwards. Event indexing is possible using either visual or inertial sensing, with the latter potentially providing system portability. To achieve maximum performance in event indexing, multi-sensor data integration is implemented, where data from both sensors is merged to automatically index key tennis events. A complete event retrieval system is also presented to allow coaches to build advanced queries which existing sports coaching solutions cannot facilitate without an inordinate amount of manual indexing

Multi-sensor classification of tennis strokes

In this work, we investigate tennis stroke recognition using a single inertial measuring unit attached to a player’s forearm during a competitive match. This paper evaluates the best approach for stroke detection using either accelerometers, gyroscopes or magnetometers, which are embedded into the inertial measuring unit. This work concludes what is the optimal training data set for stroke classification and proves that classifiers can perform well when tested on players who were not used to train the classifier. This work provides a significant step forward for our overall goal, which is to develop next generation sports coaching tools using both inertial and visual sensors in an instrumented indoor sporting environment

Combining inertial and visual sensing for human action recognition in tennis

In this paper, we present a framework for both the automatic extraction of the temporal location of tennis strokes within a match and the subsequent classification of these as being either a serve, forehand or backhand. We employ the use of low-cost visual sensing and low-cost inertial sensing to achieve these aims, whereby a single modality can be used or a fusion of both classification strategies can be adopted if both modalities are available within a given capture scenario. This flexibility allows the framework to be applicable to a variety of user scenarios and hardware infrastructures. Our proposed approach is quantitatively evaluated using data captured from elite tennis players. Results point to the extremely accurate performance of the proposed approach irrespective of input modality configuration

TennisSense: a platform for extracting semantic information from multi-camera tennis data

In this paper, we introduce TennisSense, a technology platform for the digital capture, analysis and retrieval of tennis training and matches. Our algorithms for extracting useful metadata from the overhead court camera are described and evaluated. We track the tennis ball using motion images for ball candidate detection and then link ball candidates into locally linear tracks. From these tracks we can infer when serves and rallies take place. Using background subtraction and hysteresis-type blob tracking, we track the tennis players positions. The performance of both modules is evaluated using ground-truthed data. The extracted metadata provides valuable information for indexing and efficient browsing of hours of multi-camera tennis footage and we briefly illustrative how this data is used by our tennis-coach playback interface

Multi-sensor classification of tennis strokes

In this work, we investigate tennis stroke recognition using a single inertial measuring unit attached to a player’s forearm during a competitive match. This paper evaluates the best approach for stroke detection using either accelerometers, gyroscopes or magnetometers, which are embedded into the inertial measuring unit. This work concludes what is the optimal training data set for stroke classification and proves that classifiers can perform well when tested on players who were not used to train the classifier. This work provides a significant step forward for our overall goal, which is to develop next generation sports coaching tools using both inertial and visual sensors in an instrumented indoor sporting environment

Game, shot and match: Event-based indexing of tennis

Identifying events in sports video offers great potential for advancing visual sports coaching applications. In this paper, we present our results for detecting key events in a tennis match. Our overall goal is to automatically index a complete tennis match into all the main tennis events, so that a match can be recorded using affordable visual sens-ing equipment and then be automatically indexed into key events for retrieval and editing. The tennis events detected in this paper are a tennis game, a change of end and a tennis serve - all of which share temporal commonalities. There are of course other events in tennis which we aim to index in our overall indexing system, but this paper focuses solely on the aforementioned tennis events. This paper proposes a novel approach to detect key events in an instrumented tennis environment by analysing a players location and the visual features of a player

A sensing platform for physiological and contextual feedback to tennis athletes

In this paper we describe our work on creating a multi-modal sensing platform for providing feedback to tennis coaches and players. The platform includes a fixed installation around a tennis court consisting of a video camera network and a localisation system as well as wearable sensing technology deployed to individual athletes. We describe the various components of this platform and explain how we can capture synchronised multi-modal sensor data streams for games or training sessions. We then describe the content-based retrieval system we are building to facilitate the development of novel coaching tools. We provide some examples of the queries that the system can support, where these queries are chosen to be suitably expressive so as to reflect a coach's complex information needs regarding tennis-related performance factors

TennisSense: a multi-sensory approach to performance analysis in tennis

The TennisSense Project, that is run in collaboration with Tennis Ireland, aims to create the infrastructure required to digitally capture physical, tactical and physiological data from tennis players in order to assist in their coaching and improved performance. This study examined the potential for using Wireless Inertial Monitoring Units (WIMU) to model the biomechanical aspects of the tennis stroke and for developing coaching tools that utilise this information. There is significant evidence in the current literature that the ability to accurately capture and model the accelerations, angular velocities and orientations involved in the tennis stroke could facilitate a major step forward in the application of biomechanics to tennis coachin